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ABSTRACT 



This report describes one of a series of pilot studies 
conducted to evaluate the instructional effectiveness of the "Harcourt 
Reading" program. This particular study focused on 7 kindergarten classes 
that started the study in the fall of 1999 and continued throughout the 
entire academic year. Students were taught selected themes using the 
resources and procedures contained in the "Harcourt Reading" program. 

Teachers received training from experienced consultants in methods of 
implementing the program. Instruction included a broad sample of reading 
skills from the strands of decoding, vocabulary, and comprehension. Students 
were administered the Harcourt program’s "Reading Inventory" as a pretest and 
posttest. Results indicated: (1) average scores increased consistently and 

increases were statistically significant from the beginning of the year to 
the end of the year; (2) increases rose at a sharper rate in the second half 
of the year, suggesting that the longer the students remained in the program 
the more their scores increased at a higher rate, and also suggesting that 
teacher familiarity may have played a part; and (3) students would have 
almost certainly shown a greater increase; however, a ceiling effect with 
many students achieving near perfect scores on the posttest may have 
prevented an even more significant score increase. (Contains 2 tables and a 
figure of data.) (SR) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Instructional Effectiveness of the Harcouri Reading Program 



A Study of the Instructional Effectiveness of the Harcourt 
Reading Program at the Kindergarten Grade Level 

This report describes one of a series of pilot studies that was conducted to evaluate the 
instructional effectiveness of the Harcourt Reading program. This study reports the results for 
kindergarten grade children only. 

Background Information 

Harcourt School Publishers asked the Educational Research Institute of America (ERIA) to 
conduct a series of independent pilot studies to evaluate the effectiveness of the Harcourt 
Reading program. Harcourt School Publishers sought out volunteer teachers to participate in the 
study; the Department of Test Services assisted in designing, securing, and scoring the data 
collection instruments; and ERIA conducted the study and analyzed the data that were collected. 
The study described in this report was conducted during the 1 999-2000 school year. 

Research Questions 

The following research questions guided the design, procedures, and data analysis of the pilot 
study: 



Is the Harcourt Reading program instruct ionally effective? Do selected themes 
significantly increase students ’ understanding of key reading skills, concepts, and 
strategies as measured by the program 's Reading Inventory? As measured by the 
program ’s Reading Skills Tests? 

Design and Procedures of the Study 

This particular study was focused on kindergarten students and was a continuation of a study 
started in the fall of 1999. In that study, a total of 40 teachers in Grades K, 1,2, and 3 
participated. The results of that study can be found in ERIA Report No. 101. This study includes 
only the kindergarten classes that started the study in the fall of 1999 and continued throughout 
the entire academic year. 

All of the kindergarten teachers volunteered to participate in the study. None of them had used 
the program previously. The teachers were encouraged to select two themes to pilot over a nine- 
to twelve-week period during the fall semester of the 1999-2000 academic year. In the spring 
semester they were asked if they would like to continue with the program; 7 of 12 of the classes 
continued until the second semester. One teacher left the study due to a pregnancy leave. Others 
decided not to continue because of commitments to other programs in their school district. All of 
the teachers agreed to administer data-collection instruments before beginning instruction and 
again after completing instruction. 
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The study reports on a total of 7 classrooms of students who continued in the Harcourt Reading 
program for an entire academic year. There were a total of 1 80 students in the original 
kindergarten sample. However, all pretest and posttest scores are available for only 131 of the 
1 80 students. Some students were absent from either the pretest or the posttest. Other students 
transferred to other schools. 

A quasi-experimental pretest-posttest design was used. Before instruction began, students were 
administered the Reading Inventory. The same inventory was administered as both a mid-year 
and an end-of-year posttest. The kindergarten teachers administered all tests. 

Following the pretests, the teachers taught the selected themes using the resources and 
procedures contained in the Harcourt Reading program. Teachers also received training from 
experienced consultants in methods of implementing and using the program. The Harcourt 
Reading program was used by all of the teachers. Instruction included a broad sample of reading 
skills from the strands of decoding, vocabulary, and comprehension. 

Upon completion of the selected themes, students were administered the posttests. All data- 
collection instruments were returned to the Educational Research Institute of America, where 
they were processed. The Harcourt Kindergarten Reading Inventories and all of the data were 
analyzed by ERIA. 

Findings 

Descriptions of each of the assessments used in the study and the assessment results are reported 
below. 



Content of the Kindergarten Reading Inventory 


Subtest 


Number of 




Items 


Phonemic Awareness 




Detecting Rhyming Sounds 


5 


Detecting Beginning Sounds 


5 


Detecting Ending Sounds 


5 


Graphophonemic Knowledge 




Letter Recognition 


5 


Letter-Sound Relationships: Beginning Sounds 


5 


Letter-Sound Relationships: Ending Sounds 


5 


Word Recognition 


5 


Writing (Spelling) 




Writing Letters 


5 


Writing Words 


5 


Reading Comprehension 




TOTAL 


45 
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Kindergarten Test Results 

Table 1 summarizes the pretest and posttest means and standard deviations for the Kindergarten 
Reading Inventory. 

TABLE 1 



Kindergarten Descriptive Results 



Kindergarten 


Reading Inventory (45 test items) 




Lowest Score 


Highest Score 


Mean Score 


Standard 

Deviation 


Pretest (beginning 
of year) 


3 


44 


25.77 


9.09 


Mid-year test 


7 


43 


29.42 


9.82 


Posttest (end of 
year) 


19 


45 


39.18 


6.26 



The mean scores from the beginning of the year on the Reading Inventory pretest to the midyear 
assessment improved from 58% correct to 66% correct after instruction. The mean scores at the 
end of the year were 88% correct. 

The increase from the beginning of the year to mid-year was statistically significant ( t=5.161 , 
p<.0001). The increase from the beginning of the year to the end of the year was also significant 
(t=14.25, p<.0001). 

The percent of gains on each of the tests is shown in Table 2. 

TABLE 2 



Summary of Test Score Increases 





Percent Correct and Raw Scores 


Percent Gain and Raw Score 
Gain 




Correct 

on 

Beginning 

-of-Year 

Pretest 


Correct 
on Mid- 
year 
Test 


Correct 
on End- 
of-Year 
Posttest 


Gain for 
First 
Half- 
Year of 
study 


Gain for 
Second 
Half- 
Year of 
Study 


Gain for 
Full Year 
of Study 


Kindergarten 
Reading 
Inventory (Total 
possible = 45 
points) 


58% 


66% 


88% 


+8% 


+22% 


+30% 


Mean Raw 
Scores 


25.77 


29.42 


39.18 


+3.65 


+8.42 


+13.41 
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Figure 1 provides a clear picture of the dramatic change in the range of scores from the pretest to 
the posttest. The gray bar shows an almost normal distribution of scores on the pretest with a few 
scores at the 0 to 9 (lowest) range and a few scores at the 36 to 45 (highest) range. However, after 
instruction the distribution is now skewed to the right with no scores in the lowest two ranges (0 
to 9) and (10 to 1 8) and about 70% of the scores in the highest range (36 to 45). 



Figure 1 

Pretest Frequencies Compared 
to Posttest Frequencies 
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Summary of Results 

• The average scores increased consistently and the increases were statistically significantly 
from the beginning of the year to the end of year for these kindergarten children. 

• Increases rose at a sharper rate in the second half of the year than in the first half of the year, 
suggesting that the longer the students remained in the program the more their scores 
increased at a higher rate. Teacher familiarity with the program may have also played a part 
in the more dramatic increases in the second half of the year. 

• Gains of this magnitude for only one year of instruction are quite remarkable considering that 
the teachers volunteered to teach the units and did not receive any extra training. 

• The kindergarten students would have almost certainly shown a greater increase. However, a 
ceiling effect with many students receiving near perfect scores on the posttest may have 
prevented an even more significant score increase. 

• The significant increase for these kindergarten children emphasizes the importance of a solid 
foundation at this level. These children can, and did, increase their beginning reading skills 
significantly and consistently. 
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